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ABSTRACT 



This issue of "Policy Forum" compares the mathematics and 
science achievement of students midway through elementary school, midway 
through lower secondary school, and at the end of upper secondary school. The 
Third International Mathematics and Science Study (TIMSS) , conducted in 
1995-96, is the largest international education study ever undertaken with 
data from one-half million students in 41 nations. The featured article 
provides significant research findings from TIMSS on student achievement, 
curriculum, and teaching, and discusses implications for policy and practice. 
The concept of benchmarking is reviewed in a sidebar. Information on how to 
access the U.S. Department of Education, the two Research and Development 
(R&D) Centers funded by this institute, and a variety of TIMSS Web sites are 
also provided. (ASK) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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A Wake-Up Call for U.S. Educators: 

The Third International Mathematics and Science Study 



The Third International Mathematics and Science 
Study (TIMSS) is the largest international education 
study ever undertaken with data from one-half 
million students in 41 nations. Conducted in 1995- 
96 under the auspices of the International Associa- 
tion for the Evaluation of Education Achievement 
(IEA), it is also one of the most rigorous and compre- 
hensive international studies. 



TIMSS compares the mathematics and science 
achievement of students midway through elemen- 
tary school, midway through lower secondary 
school, and at the end of upper secondary school. In 
the United States, we tested over 33,000 students in 
fcnore than 500 schools in the 4th, 8th, and 12th 
grades. As well as content assessments, there were 
curriculum analyses, teacher surveys, and a video 
study of teaching in three countries. 



Significant Research Findings 

TIMSS offers significant findings in three key areas: 
student achievement, curriculum, and teaching. 

Student Achievement. American school children 
were the only students with above average scores in 
4th grade to lose ground in the 8th by testing aver- 
age, and then to do worse again in the 12th. By the 
time our students had moved through our education 
system they were among the lowest scoring students 
in the study. 



The 4th-grade achievement of American students is 
quite high. Our students are near the top in both 
mathematics and science. Of the 25 participating 
countries, our 4th-graders were outperformed in sci- 
ence only by Korea; showed no significant difference 
with 5 other countries; and outperformed 19 other 
countries. In mathematics, our 4th-graders were out- 
4 - performed by 7 countries; 6 countries were not sig- 

P nificantly different; and our students did better than 
12 nations. 
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In middle school (8th grade in the United States), 41 
countries participated. While our students were 
above the international average in science, they were 
below the international average in mathematics. 
American students were outperformed in science by 
9 nations; performed similarly to 17 nations; and out- 
performed 15 nations. Twenty nations outper- 
formed American students in 8th-grade 
mathematics; 14 performed at the same level; and 7 
nations fell below our students. 



In a pattern similar to 4th and 8th grade, the United 
States' 12th-grade international standing in science 
is higher than in mathematics. However, both U.S. 
science and mathematics performances are below 
the international average and among the lowest of 
the 21 TIMSS countries. In science, students in 11 
countries outperformed American students; stu- 
dents in 7 countries performed similarly to Ameri- 
can students; and American students outperformed 
students in only 2 countries. In mathematics, Ameri- 
can 12th-graders performed below the international 
average and among the lowest of the 21 TIMSS coun- 
tries. American students were outperformed by 14 
countries; were not different than 4 countries; and 
outperformed students in 2 countries. Even our best 
mathematics and science students did not fare well 
in comparison with the best mathematics and sci- 
ence students from other countries. 



Curriculum. There are no required national curricu- 
lum standards in the United States. Curriculum 
standards are developed and maintained at various 
levels of government — local, state, and national. 

The TIMSS curriculum studies were conducted by 
researchers at Michigan State University. It appears 
that U.S. mathematics and science curricula lack the 
coherence, focus, and rigor of the curricula taught in 
other countries that participated in TIMSS. Most 
state curriculum frameworks emphasize breadth 
Over depth. (Continued on page 6) 
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What Is Benchmarking? 



While benchmarking is unfamil- 
iar to most educators, it has been 
successfully used in business to 
foster continuous improvement 
and achieve excellence. Bench- 
marking can help organizations — 
including schools, districts, and 
state agencies — become high per- 
formance learning communities. 

Benchmarking is an activity 
where organizations continu- 
ously engage in self-study and 
compare themselves with the 
leaders in their field so they can 
identify, adapt, and apply sig- 
nificantly better practices. 

It is a systematic, evidence- 
based, and participatory change 
process. Benchmarking may be 
done at the strategic level where 
the goal is to create a shared vi- 
sion and identify key levers for 
organizational change. If the stra- 



tegic thinking has already been 
done, then benchmarking can 
take place at the level of specific 
practices or processes (e.g., im- 
prove curriculum or professional 
development). 

The benchmarking process in- 
cludes the following steps: 

• Self-assessment. Document 
and study your own organiza- 
tion's vision, practices, and 
success measures. Decide 
what to benchmark. 

• Comparison. Decide who to 
benchmark. Identify exem- 
plars and establish a bench- 
marking partnership. Study 
and assess your partner(s). 

For example, in the areas of 
mathematics and science edu- 
cation, you can use local exem- 
plars as well as TIMSS data for 
comparison purposes. 



• Analysis and Adaptation. 

Ask why you are getting your^^ 
results and why others are g&W 
ting better results. While 
benchmarking is often called 
"borrowing shamelessly," 
practices generally require 
creative adaptation in a new 
context. 

• Implementation. Think care- 
fully about what enablers 
(e.g., resources, schedule 
changes) are needed. Commu- 
nicate findings and build sup- 
port for the changes you want 
to make. 

• Feedback. Carefully monitor 
and measure the results of 
your innovation and recali- 
brate if necessary. 



Benchmarking Resources 

Read more about the benchmarking process. These references were used as sources for this article. 

Tucker, Sue. 1996. Benchmarking: A Guide for Educators. Thousand Oaks, CA: Corwin Press. A brief introduction 
adapted for educators in a clear "how to" format. 

Resnick, Lauren B. and Nolan, Katherine J. 1995. "Benchmarking Education Standards." Education Evaluation and Policy 
Analysis, 17(4), 438-61. An example of an international benchmarking study in education. 

Bogan, Christopher E. and English, Michael J. 1994. Benchmarking for Best Practices: Winning through Innovative Adapta- 
tion. New York: McGraw Hill. An in-depth treatment written for the business context. 

Watson, Gregory H. 1993. Strategic Benchmarking: How to Rate Your Company's Performance Against the World's Best. New 
York: John Wiley and Sons. A classic in the field. 

Use existing data for international benchmarks in math and science education. 

Government Printing Office, 1997. Attaining Excellence: A TIMSS Resource Kit. Designed to help educators use TIMSS, 
the kit contains multimedia materials including reports, videotapes, and discussion overheads. It includes four mod- 
ules: overview, achievement, curriculum, and teaching. Call GPO at (202) 512-1800 to purchase the whole kit in a 
carry-case box for $94. (Modules also available separately.) The kit can also be downloaded free from the Internet 
(see p. 4). 

Stigler, James W. and Stevenson, Harold W. Spring 1991. "How Asian Teachers Polish Each Lesson to Perfection," 
American Educator, 12-20, 43-47. This article compares key features of elementary school mathematics lessons in 
Taiwan, Japan, and the United States. With a related 34-minute videotape depicting classroom scenes, these materi- 
als may be used to introduce international benchmarking to a variety of audiences, especially elementary mathemat- 
ics teachers. The videotape may be ordered by sending a check for $35 to the University of Michigan, c/ o Mrs. S. 
Liang, 30 N. Ingalls, 10th Floor, Ann Arbor, MI 48109. 
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TIMSS: A " 



Power Tool" for 



School 



Reform 



In October 1997, the National In- 
^^titute on Educational Govem- 
^pice, Finance, Policymaking, and 
Management sponsored a TIMSS 
Policy Forum to explore the prac- 
tical uses of TIMSS. These inter- 
views with a few Forum 
participants illustrate how 
TIMSS may be used by state 
agencies, school districts, profes- 
sional associations, and 
researchers. 




Susan Zelman, Deputy Commissioner, 
Department of Elementary and 
Secondary Education, Missouri 



We want to move toward a new 
educational paradigm in Mis- 
souri. Our vision is to replace 
classrooms where teachers talk 
and students passively listen 
with classrooms where teachers 
facilitate hands-on learning ac- 
tivities and students learn to 
think and apply their knowledge. 

As we pursue this goal, we have 
used TIMSS to strengthen our 
state assessment program and to 
enhance our professional devel- 
opment efforts. 



We administered the TIMSS test 
to about 12,000 students. The 
test results were used for a con- 
tent validation of the eighth- 
grade mathematics test in our 



Missouri Assessment Plan. We 
also used the TIMSS test results 
to benchmark our curriculum. It 
helped us identify specific areas 
where we were doing well and 
others where we needed to im- 
prove. Also, we found the attitu- 
dinal data on teachers and 
students very helpful. For exam- 
ple, data on why students think 
they succeed offers insight about 
student motivation. 

Professional development is a 
key reform strategy in Missouri. 
We have established nine re- 
gional professional development 
centers based in state colleges 
and universities where we train 
lead teachers in authentic instruc- 
tion and assessment methods. 

The TIMSS materials, especially 
the videos, have been used suc- 
cessfully with teachers in our 
centers. 




Paul Kimmelman, Superintendent, 
West Northfield School District #31 in 
Illinois and President, First in the World 
Consortium 

The First in the World Consor- 
tium was created 3 V 2 years ago 
and now includes 19 suburban Il- 
linois school districts. Our Con- 
sortium is deeply committed to 
National Education Goal #5 — be- 
coming first in the world in 



mathematics and science. Be- 
cause we saw the potential of 
benchmarking with TIMSS as an 
important means to our ambi- 
tious goal, we formed a partner- 
ship with the U.S. Department of 
Education which enabled us to 
participate in TIMSS as if we 
were a country. A total of 3,100 
students were tested repre- 
senting a student population of 
37,000° 

Now we are creating learning 
communities to study the TIMSS 
results and compare our achieve- 
ment with other countries. The 
rich data about curriculum and 
teaching are generating ideas for 
improved practice in our dis- 
tricts. We have supported two 
summer institutes with 350 par- 
ticipants, including teachers, ad- 
ministrators, and school board 
members. This year, institute 
participants will develop curricu- 
lum products that exemplify 
new concepts of teaching and 
learning in math and science. 
Throughout the year, teachers 
across our districts are involved 
in learning networks which sup- 
port collaboration around cur- 
riculum development based on 
our TIMSS data. 

We have partnered with the 
North Central Regional Educa- 
tional Laboratory (NCREL) for 
assistance in planning, docu- 
menting, and disseminating our 
activities. Further information 
about our consortium is avail- 
able on the Internet 
(http://zvzvw.ncrel.org). 

'Editor's Note: The National Center for 
Education Statistics has made it possible 
through the 1999 TIMSS Benchmarking for 
States and Districts study for more states 
and school districts to participate directly in 
international benchmarking. This spring 
about 30 states, districts, and/or consortia 
will collect data that will enable comparisons 
with 40 countries. 
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Alice Gill, Associate Director, 
Educational Issues Department, 
American Federation of Teachers (AFT) 



TIMSS is very important at AFT 
because it supports our long- 
standing emphasis on high stand- 
ards. We see it as a valuable tool 
for benchmarking U.S. teaching 
practices, curriculum, and educa- 
tional policy with the best in the 
world. We have developed an 
AFT "TIMSS Kit" to help our 
members understand what the 
findings say and what they don't 
say. We also have incorporated 
the data and the videotape into 
our "Thinking Mathematics" re- 
search synthesis and professional 
development for elementary and 
middle school teachers. 

We have introduced thousands 
of teachers and union leaders to 
the TIMSS materials at various 
meetings, including the "Lessons 
from the World" TIMSS Confer- 
ence that AFT co-sponsored with 
the National Center for Educa- 
tion Statistics in February 1997, 
the Summer and Winter Insti- 
tutes of the Educational Research 
and Dissemination Program, and 
regional meetings for "Thinking 
Mathematics" teacher leaders. 
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These members, in turn, dissemi- 
nate the findings to many other 
groups. 

Teachers find the TIMSS 
videotape study extremely useful 
as a focus for improving their 
mathematics and science lessons. 
They use it, for example, to exam- 
ine the level of challenge in their 
questioning, the coherence of 
their lessons, and the focus and 
content of their curricula. 
Videotape is a powerful tool for 
reflection and the TIMSS data are 
rich with information beyond the 
"horse race." 




Susan Fuhrman, Director, Consortium 
for Policy Research in Education 
(CPRE), University of Pennsylvania 



Photo courtesy of Tommy Leonardi 

The TIMSS data offer exciting op- 
portunities for educational re- 
searchers. I can see at least four 
important lines of inquiry these 
data can support: multivariate 
analyses of factors contributing 
to student achievement; cross-na- 
tional research, especially in the 
areas of curriculum, instruction, 
and assessment; studies of 
effective strategies for improving 
math and science education; and 
in-depth studies of teaching 



practice, an area where TIMSS ex- 
emplifies significant methodo- 
logical advances. 

At the University of Pennsylva- ^ 
nia, we are already engaged in 
TIMSS-related research. With Of- 
fice of Educational Research and 
Improvement (OERI) and Na- 
tional Science Foundation (NSF) 
funding, we will be conducting a 
comprehensive multivariate 
analysis to address this question: 
Why did American 8th- and 12th- 
graders perform so poorly on 
TIMSS science and math tests, 
and why was there such a sharp 
deterioration of performance 
from the 4th- to 12th-grades? 

Also with OERI funding, CPRE 
is sponsoring three TIMSS Fo- 
rums between February 1999 
and June 2001 to help identify im- 
portant issues for the next itera- 
tion of this international study 
and to facilitate dissemination of 
information from this round. 



— Norma Fleischman of the Insti- 
tute staff contributed to this article. 



Publication Available 

For those who would like more 
information about our TIMSS 
Policy Forum, a free booklet 
summarizing the discussion is 
available. Call (877) 4ED-PUBS 
(877-433-7827) and ask for Pol- 
icy Brief: What the Third Interna- 
tional Mathematics and Science 
Study (TIMSS) Means for Sys- 
temic School Improvement. Also 
look for it on the Internet 
(http:l I www. ed.gov/pubs/ 
edpubs.html). 
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• schools and other education 
organizations "get wired" and 
researchers put their findings on 
the Web, the Internet may help 
close the gap between education 
research and practice. Many of 
our readers have expressed an 
interest in getting education re- 
search on the Internet. Informa- 
tion on how to access the U.S. 
Department of Education, the 
two R&D Centers funded by this 
Institute, and a variety of TIMSS 
Web sites follows. Enjoy surfing! 

U.S. Department of Education 
(ED) has a home page 
(http://zvzvzv.ed.gov). It offers a 
wide variety of information 
about federal education pro- 
grams, priorities, and events. To 
get information specifically 
about research, statistics, and as- 
sessment, go to the blue sidebar 
\ on the left and click on "Research 
and Statistics." Scroll down the 
page and you will see the Na- 
tional Institute on Educational 
Governance, Finance, Policymak- 
ing, and Management listed un- 
der "Research Institutes" (along 
with the four other Institutes). 

By clicking on our name, you can 
get information about us or you 
can link to the home pages of the 
two national R&D Centers we 
fund which are described below. 



Consortium for Policy Research 
in Education (CPRE), a grantee 
of this Institute, recently 
expanded its Web site which fea- 



WWW. Education Research 

tures policy research. Highlights 
include: center mission and par- 
ticipating universities, detailed 
information about current CPRE 
research projects, publications 
information, case studies, and 
a "paper of the month." CPRE 
Policy and Finance Briefs can 
be downloaded. You can access 
the Web site directly at 
http://zvzvzv.upenn.edu/gse/cpre 
or contact Robb Sewell at (215) 
573-0700, ext. 225 for further 
details. 

Center for the Study of Teach- 
ing and Policy (CTP) recently re- 
ceived a grant from this Institute 
to investigate the relationship be- 
tween excellent teaching and 
policymaking. The Center's re- 
search is still in the beginning 
stages, but information about 
findings will be added as it be- 
comes available. Information 
about the Center's mission and 
research plans is on their Web 
site. The address is http:/ / depts. 
zvashington.edu/ctpmail or for 
further information contact 
Michele Ferguson at (206) 221- 
4114. 

Third International Mathemat- 
ics and Science Study (TIMSS) 
information can be obtained 
through the main ED Web site 
described above or use 
http:llnces.ed.gov/timss for 
direct access. Here you can 
download the major TIMSS 4th-, 

8th-, and 12th-grade studies. For 



additional information, call the 
TIMSS Customer Service Line at 
(202) 219-1333. 

Several nongovernment Web 
sites also offer information about 
TIMSS. 

International Association for 
the Evaluation of Educational 
Achievement has information 
about TIMSS as well as other 
international studies on such 
topics as technology and reading 
(h ttp:l / zvzvzv.iea.nl/). 

Boston College provides links 
to a variety of other TIMSS 
Web sites in the United 
States and other nations 
(http:llzvzvzv.csteep.bc.edu/Timssl 
lTIMSSSites.html) and also 
has details about the TIMSS 
database, including data files, 
questionnaires, and database 
user guides (http://zvzvzv.csteep. 
bc.edu/timssl/database.html). 

Michigan State University 
has information about the 
TIMSS curriculum study 
(h ttp://ustimss. msu.edu/) . 

Eisenhower National Clearing- 
house is the source for the 
TIMSS Resource Kit which can 
be downloaded for free. In addi- 
tion to TIMSS information, this 
site includes other tools useful 
for benchmarking as well as 
national and state standards 
documents ( http :// timss.enc.org/). 



Please send your comments and suggestions for the newsletter to: 

National Institute on Educational Governance, Finance, 
Policymaking, and Management 
Newsletter Editor 

555 New Jersey Avenue NW, Room 608 
Washington, DC 20208-5510 

FAX: (202) 219-2159 
E-mail: gfi@ed.gov 
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( A Wake-Up Call, continued from page 1) 

Mathematics and science textbooks in the United 
States were found to be substantially longer than the 
international average and to express the incoher- 
ence, fragmentation, and lack of rigor noted in the 
U.S. curriculum. 1 

Teaching. The 8th-grade mathematics videotape 
classroom study conducted in the United States, Ger- 
many, and Japan found American and German les- 
sons, unlike Japanese lessons, to focus primarily on 
the acquisition and application of skills rather than 
problem solving and thinking. When mathematical 
concepts are addressed, three-fourths of Japanese 
and German teachers developed the concepts after 
introducing them compared to less than one-fifth of 
American teachers. American teachers tend only to 
state rather than develop concepts. There was less co- 
hesion in American versus Japanese.Iess.ons. The av- 
erage grade level of 8th-grade ma thema t-igsitasa ght , 1 ^ - - 
the United States was 7.4 versus' &£irr Germany and 
9.1 in Japan. While 62 percent of' Japanese and 21 
percent of German 8th-grade mathematics lessons in- 
cluded deductive reasoning, no American lessons 
did. 

Lesson plans from each country (with national ori- 
gin disguised) were judged by an independent . 
group of American college mathematics teachers. 
Eighty-nine percent of American lessons were found 
to be of low quality and none were judged to be of 
high quality. In comparison, 11 percent of Japanese 
lessons and 34 percent of German lessons were 
found to be of low quality, and 39 percent of Japa- 
nese lessons and 28 percent of German lessons were 
judged to be of high quality. 

’Schmidt, W.H., McKnight, C.C., and Raizen, S.A., et al. 1997. The Splin- 
tered Vision: An Investigation of U.S. Science and Mathematics Education. 
Norwell, MA: Kluwer Academic Publishers. 



Implications for Policy and Practice 

TIMSS is not an answer book, but it does provide us 
a new lens through which to view our education sys- 
tem. These findings challenge aspects of American 
education that we have previously taken for granted| 
and force us to reevaluate our cultural assumptions 
about educational excellence. By comparing our 
education system to those of other countries, we bet- 
ter understand ourselves. Careful reflection on our 
education system within this international perspec- 
tive can provide valuable insights for educators, 
teachers, parents, and business leaders as we move 
our nation forward in the pursuit of educational re- 
form. 



Obviously, we have much work to do to improve 
student achievement .in the upper grades. , We do 
give our students' i 1 good start, but wefffuSt urgently 
ask ourselves why-U:S. students lose the lead as they 

ence content affJFgrade four/ W e" cannot afford to 
be complacent about our curriculum and teaching 
practices. The TIMSS data warn that our bench- 
marks in these areas need to be much higher. Re- 
formers who choose to use the information available 
in TIMSS for benchmarking toward higher stand- 
ards have an unprecedented opportunity to improve 
American mathematics and science education . 



— Douglas Cochrane, Education Statistics Services 
Institute. 



Editor's Note: TIMSS was initiated and funded primarily by the National 
Center for Education Statistics (NCES), with supplementary support for 
international coordination from the National Science Foundation (NSF) 
and the Canadian Government. 
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